fix(metrics): detect key collisions between dimensions, metrics, and metadata#5240
fix(metrics): detect key collisions between dimensions, metrics, and metadata#5240Zelys-DFKH wants to merge 7 commits into
Conversation
|
Thanks for tackling this! Two pieces of feedback: 1. Only throw on type changes Same-type collisions (dimension ↔ metadata, both strings) overwrite silently but don't corrupt the EMF schema — CloudWatch only rejects when a key flips between string and number. Throwing on same-type clashes is stricter than the spec requires and inconsistent with 2. Centralise the check in The current guards in
Reverse the call order in either of the existing tests and the corruption comes back. Rather than adding guards to every mutator, // inside serializeMetrics(), just before the return
type Source = 'dimension' | 'default dimension' | 'dimension set' | 'metadata';
const stringKeys = new Map<string, Source>();
for (const k of Object.keys(defaultDimensions)) stringKeys.set(k, 'default dimension');
for (const k of Object.keys(dimensions)) stringKeys.set(k, 'dimension');
for (const set of dimensionSets) for (const k of Object.keys(set)) stringKeys.set(k, 'dimension set');
for (const k of Object.keys(this.#metadataStore.getAll())) stringKeys.set(k, 'metadata');
for (const name of Object.keys(metricValues)) {
const source = stringKeys.get(name);
if (source) {
throw new Error(
`EMF key collision on "${name}": registered as both a metric (number) and a ${source} (string)`
);
}
} |
Per reviewer feedback: move the type-change collision guard from individual mutators (storeMetric, addMetadata) to serializeMetrics(), where all string and numeric sources are in scope simultaneously. The prior approach only caught two of the four collision directions (metric-after-dimension and metadata-after-metric). The centralized check catches all four regardless of call order, and throws only on type-change collisions (metric number vs. string source) not same-type overwrites, which CloudWatch handles silently. New error format: 'EMF key collision on "<key>": registered as both a metric (number) and a <source> (string)' where source is 'dimension', 'default dimension', 'dimension set', or 'metadata'. Tests updated to assert at serializeMetrics() time; two reverse-direction tests added (dimension-after-metric, metadata-before-metric). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Hi @Zelys-DFKH, do you need any help with this PR? Anything you need me to clarify from my comment? |
|
Sorry for the slow response, I missed the notification. Both points were right. Pushed a second commit that removes the guards from Thanks for the ping. |
|
No worries! Couple of comments, the way the code is now, we only warn on dimesnions overwriting each other but we should warn whenever any keys are overwritten. Secondly, there are merge conflicts in the test file that need to be resolved. Also, the |
…ision check - Pull collision detection out of serializeMetrics() into a private #checkKeyCollisions() method, addressing the cyclomatic complexity warning flagged by SonarCloud - Extend the check to warn (via the logger) when a metadata key would silently overwrite a dimension, default dimension, or dimension set value in the serialized EMF output; metric collisions still throw - Add three regression tests in metadata.test.ts covering the new metadata-vs-dimension-source warning path Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
| metricValues: Record<string, number | number[]>, | ||
| metadata: Record<string, string> | ||
| ): void { | ||
| type Source = |
There was a problem hiding this comment.
Let's not inline types, there's a types module for this.
…pstream Extends #checkKeyCollisions to warn whenever any two string sources share a key (dimension overwriting default dimension, dimension set overwriting dimension or default dimension), not just when metadata overlaps with a dimension. Folds the repeated get+check+set pattern into a single setKey helper. Also reconciles dimensions.test.ts with upstream aws-powertools#5229 (MAX_DIMENSION_COUNT loop bounds, toThrow vs toThrowError, three new upstream test cases) and adds regression tests for the three new cross-source overwrite warn paths. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Thanks for the thorough review — pushed b2a61fa.
Let me know if anything else needs a look! |
|
Left some comments, also looks like we still have merge conflicts |
…over metadata Three changes addressing PR review feedback: 1. EmfKeySource extracted from the inline type alias in #checkKeyCollisions to packages/metrics/src/types/Metrics.ts. 2. Dimensions now take precedence over metadata in the serialized EMF. The prior spread order put metadata last, so a metadata value silently overrode any dimension on the same key, contradicting the warning that said the dimension would win. The fix flips the spread (metadata first, then dimensions) and reorders the seed in #checkKeyCollisions so the warning message matches what actually ends up in the payload. 3. Collision tests moved from dimensions.test.ts to a new keyCollisions.test.ts. dimensions.test.ts is now byte-identical to upstream/main, which clears the merge conflict GitHub flagged after aws-powertools#5229 merged. Also drops the now-unused DimensionsStore.getDimensionCount(), which aws-powertools#5229 stopped calling. Test count: 156 unit tests pass.
|
Thanks for the careful review @svozza. Pushed 3e92ae8 with all three points addressed.
All checks green. |
| import { Metrics, MetricUnit } from '../../src/index.js'; | ||
|
|
||
| describe('EMF key collision detection', () => { | ||
| const ENVIRONMENT_VARIABLES = process.env; |
There was a problem hiding this comment.
Let's use the built in Vitest environment variable functions here: https://vitest.dev/api/vi.html#vi-stubenv.
There was a problem hiding this comment.
Deleted keyCollisions.test.ts. The 8 tests are now in dimensions.test.ts, which uses the same env setup.
| @@ -0,0 +1,138 @@ | |||
| import { beforeEach, describe, expect, it, vi } from 'vitest'; | |||
There was a problem hiding this comment.
Am I missing something, these tests look incredibile similar to the ones in metadata.test.ts?
There was a problem hiding this comment.
The tests weren't duplicating metadata.test.ts (those cover metadata/metric collisions, these cover dimension/metric), but having a separate file for it was the real issue. Moved all 8 into dimensions.test.ts and deleted keyCollisions.test.ts. Dimension/metric and dimension/dimension collision tests live there now; metadata.test.ts owns the metadata side.
Deleted keyCollisions.test.ts and moved all 8 tests into dimensions.test.ts, where they belong alongside the existing dimension behavior tests. metadata.test.ts owns the metadata-side collision tests; dimensions.test.ts now owns the dimension-side. Co-authored-by: Claude Signed-off-by: Zelys-DFKH <admin@coracreacrafts.com>
|



Summary
Closes #5208. `serializeMetrics()` builds EMF output by spreading multiple key sources in a fixed order: metadata → defaultDimensions → dimensions → dimensionSets → metricValues. When a key appears in more than one source, the later value wins silently. Two of those overwrites break the EMF schema:
The bug is call-order-dependent. A guard at the call site of `addMetric()` or `addDimension()` can only catch one direction; the fix belongs at `serializeMetrics()`, where all sources are present before the spread.
What changed
Issue number: Closes #5208
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.